APPENDIX 1: EXAMPLES OF ALGORITHMIC STEPS 



EXAMPLE IMAGE 

5 The Figure A below shows an example image on which some of the algorithmic steps 

are demonstrated. The size of this image is 20 rows by 20 columns. 




5 10 15 20 



Figure A: Example input image with 20 rows and 20 
columns. 
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The image shown above is represented as a matrix of numbers and can also be 
displayed in 20 x 20 matrix form where the black pixels on the image have a number zero 
and white pixels have a number 10 in this case: 
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To illustrate some of our algorithms, we also use a noisy version of this example 
image. To create this image, we added random values to every pixel of the example image 
and this noisy image is shown in Figure B. 
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As before, the noisy image is shown as a 20 x 20 matrix below: 
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HEAT FILTER -514 

The Laplacian (second derivative) of the example image is computed as the sum of 

d 2 u d 2 u 

the second partial x-derivative and second partial y-derivative of the image: — - + — - . This 

dx 2 dy 

35 Laplacian is the right-hand side of the heat equation El . 



66- 



25315 

PATENT TRADEMARK OFFICE 



DXUC-1-1020AP 



Black Lowe & Graham f 

^ 

816 Second Avenue 
Seattle, Washington 98104 
206.381.3300 • F: 206.381.3301 



The Laplacian of the example image is computed and this is shown in Figure C(l). 
Notice the negative values on the inside of the bright areas and positive values on the outside 
of the bright areas at the edges of the regions. The Laplacian is zero an pixels far from edges. 




(1) (2) 
Figure C (1): The Laplacian of the example image 
shown in Figure A. (2) The Laplacian added back to 
the input image. 



When this Laplacian is added back to the input image, after multiplying with a step 
size parameter, it results in blurring of the edges since the bright areas are reduced in 
brightness and the dark areas are increased in brightness - this output is shown in Figure 
C(2). This is the output of a single iteration of the heat filter. The heat filter outputs after 4 
iterations and after 20 iterations are shown in (1) (2) 

Figure D. Notice the progressive blurring of the image as you apply more iterations. 




Figure D: Heat filter outputs after 4 iterations (1) and 
after 20 iterations (2) on the image shown in Figure 
A. 
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Applying 20 iterations of the heat filter to the noisy image from Figure B results in a 
reduction of noise and a blurring of the image. This output is shown in Figure E. 




10 



Note that in this image, the noise has been significantly removed, although the image 
is very blurred. This image looks very similar to the heat filtered version of the noiseless 
image shown in Figure D(2). 

SHOCK FILTER -518 



The application of the shock filtered to the blurred image is shown in Figure E to 
sharpen this image. For purposes of the shock filter, the Laplacian of an image (Equation E4) 
15 is computed as: 

£(u) = u^u 2 + 2u xy u x u y + Uyy u 2 . 

This is computed for the blurred image and is shown below in shown in Figure F(l). 
20 Again, notice the negative values on the inside of the bright areas and positive values on the 
outside of the bright areas at the edges of the regions. 

The gradient magnitude of the blurred image (Equation E3) is: 



25 pu\\ = fi^~u 



This is computed for the blurred image and is shown in Figure F(2) below. Note that 
the gradient magnitude is highest at the edges and zero in smooth regions. 
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Using a threshold value, t, on pixel gradient to be 0.7 we can compute the F function, 
which is positive where the Laplacian is greater than zero and the gradient magnitude is 
greater than a threshold, negative where the Laplacian is less than zero and the gradient 
10 magnitude is greater than the threshold and zero elsewhere: 

F(£(u)) = 1 , if t(u) > 0 and||Vw|| > t 

= -l,if £(u)<0 and||Vw||>; 
= 0, otherwise 

15 

This F image is shown in Figure G(l) below. Note that as with the Laplacian image, 
everything outside the original white regions at the edge is 1, everything inside the original 
white regions at the edge is -1. The boundaries of the original region can be seen as the 
20 transition between the negative and the positive F values (the zero crossings). 

Now, the right hand side of the shock filter equation E2, is the negative of the product 
of the F function image Figure G(l) and the gradient magnitude image shown in Figure F(2): 
-F{t{u)pu\ 



This right hand side of the shock filter equation is shown in Figure G(2). If we add 
this image to the original blurred image we can see that it has the opposite effect of the heat 
filter. The dark pixels in Figure G(2) are subtracted from the blurred image and the bright 
pixels are added to the blurred image basically, thereby making the blurred image more crisp. 
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The addition of this image to the input blurred image is basically what constitutes a single 
iteration of the shock filter - this addition is shown in Figure H(l). After 20 iterations, a 
crisper output as shown in Figure H(2) is obtained. While this image is not identical to the 
input noiseless image of Figure A, it is quite similar to it and much more noise free as 
5 compared to the noisy version shown in Figure B. 
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(1) 

10 Figure G: (1) The F function 

Laplacian and the gradient. (2) 
product of the F function and the 
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(2) 

computed from the 
The negative of the 
gradient. 




Figure H(l) The output of 1 iteration of the shock 
15 filter on the blurred image from Figure E. (2) The 

output of 20 iterations of the shock filter on the same 
blurred image. 



INTENSITY CLUSTERING - 422 

20 
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In intensity clustering, the enhanced image is subject to the k-means clustering 
algorithm. In this example, the enhanced image, shown in Figure H(2), is clustered into 4 
clusters. 

5 The minimum and maximum intensity values are 1.69 and 13.40. We divide that 

range into 4 partitions and get the following initial cluster boundaries. 

1.19 4.12 7.04 9.98 13.90 

With this partition, all pixels with values between 1.19 and 4.12 are assigned into 
cluster 1, all between 4.12 and 7.04 are assigned into cluster 2, etc. This initial assignment of 
every pixel to the 4 clusters is shown in Figure 1(1) . Based on this assignment new cluster 
centers are found and new partition boundaries found accordingly - after this first iteration, 
the cluster boundaries move slightly and they are now at: 

1.19 4.08 6.30 9.57 13.90. 

In the next iteration, the pixel assignment based on these partition boundaries is 
shown in Figure 1(2). Finally, after 6 iterations, the partition boundaries change very little 
20 between iterations and are set to: 

1.19 3.65 5.646 9.25 13.90 

The final assignment of pixels to the 4 clusters is shown in Figure 1(3). For this 
example, the output of the clustering step is the set of pixels assigned to the brightest cluster. 
25 This output is shown in Figure 1(4). 
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(3) W 
Figure I: The output of clustering at different 
iterations of the k-means clustering process. (1) 
Initial assignment (2) After the first iteration (3) 
Final assignment after 6 iterations. (4) The one 
brightest cluster representing the region of interest. 



(4) 



SPATIAL GRADIENT - 526 



10 To compute the gradient magnitude of the enhanced image shown in Figure H(2), the 

x-derivative u x and the y-derivative u y of the image is calculated and then the sum of 

squares of the two images is calculated to get the gradient magnitude ~ Vu = Ju z +u 2 . The 



x- and y-derivatives and the gradient magnitudes of the enhanced image are shown in Figure 
J. 



15 




(1) (2) 
Figure J: Spatial gradients of the image shown in 
Figure H(2): (1) x-derivative of image (2) y- 
derivative of image (3) gradient magnitude of image. 



(3) 
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HYSTERESIS THRESHOLD - 530 
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In the hysteresis threshold, two threshold values based on the percentage of non-edge 
pixels desired and the ratio of the two thresholds is determined. In this example, the 
percentage non-edge pixels desired is set to 90% and the lower to upper threshold ratio is set 
to 0.5. 

5 

The upper threshold is first computed by looking at the histogram and the cumulative 
histogram of the gradient magnitude image shown in Figure J(3). The histogram of the 
gradient magnitude image is shown in Figure K(l). The histogram shows the count of pixels 
of different gradient values on the image. The cumulative histogram calculated by 

10 cumulatively adding up the bins of the histogram is shown in Figure K(2). The cumulative 
histogram shows the count of all pixels less than a given value in the gradient magnitude 
image. The horizontal line shown in Figure K(2) corresponds to a y-value of 90% of the 
image pixels (0.90 * 20 * 20 = 360). The intersection of this line with the cumulative 
histogram gives the threshold at which 90% of the image pixels will be marked as non-edge 

15 and 10% will be marked as edge. From the graph, we can see that a threshold of 7 satisfies 
that condition. The lower threshold is set to 0.5 times the upper threshold of 7, which gives 
3.5. 




20 Figure K: Histogram and cumulative histogram of 

the gradient magnitude image. 



For the hysteresis threshold, the threshold of the image at the two threshold values of 
3.5 and at 7 is set and then all edge segments from the lower threshold image is selected that 
25 have at least one higher threshold pixel. The two thresholded images and the output of the 
Hysteresis threshold step are shown in Figure L. In this example, note that one edge segment 
- (row 15, column 2) has no pixel greater than the higher threshold and is therefore removed 
from the output. All other edge segments from Figure L(l) are retained in the output. 
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(1) (2) 
Figure L: (1) Gradient image thresholded at 3.5 - all 
pixels greater than 3.5 are shown as white (2) 
gradient image thresholded at 7, (3) Output of 
hysteresis threshold. 



(3) 



MATCHING EDGES FILTER - 538 
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The matching edges filter selects pairs of matching edges from among all the edges 
found by the hysteresis threshold step (Figure L(3)). The y-derivative image , shown in 
Figure J(2) is used in this step to help find the vertical matching edges. The matching edges 
found by this step of the algorithm are shown in Figure M(l) and the filled region between 
the leading the trailing edges are shown in Figure M(2). Note that this step gets rid of edges 
of regions on the periphery of the image since it cannot find matching trailing edges for those 
regions. Also, note that in the output filled image we have some unfilled lines where the 
gradient was primarily horizontal or where the matching edges did not exist. 





0) (2) 
Figure M: (1) Matching edges found by the algorithm 
- leading edges shown in gray and lagging edges 
shown in white. (2) The region between the leading 
the lagging edges filled by white pixels. 
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AND IMAGES - 442 



The two images shown in Figure 1(4) and Figure M(2) represent the segmentation of 
the input noisy image shown in Figure B. Neither of them represents a perfect output. The 
region-based algorithm finds extraneous regions while the edge-based algorithm misses part 
of the desired region. The two images are combined using a Boolean AND operator and the 
output is shown in Figure N. 



Figure N: Combining the region-based and edge- 
based segmentation. 



CLOSE AND OPEN - 546 & 550 

Using a structuring element shaped like a line of width 5 pixels, we close the output 
of AND images and this gives us a single region representing the structure of interest as 
shown in Figure O. 




Figure O: Result of closing the image from Figure N. 
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Opening this image does not change the result - however, Figure P shows an example 
of opening on another image. The input image, Figure P(l), contains two regions, one large 
region and one small region. The large region is 10 pixels in width and height while the small 
5 region is 4 pixels in width and height. Using a structuring element shaped like a line of length 
5 pixels and applying morphological opening to the input image results in an image shown in 
Figure P(2) where the smaller region whose width was smaller than the structuring element 
was removed from the image. 




5 10 15 20 5 10 15 20 



10 

(1) (2) 
Figure P: (1) Input image containing two foreground 
regions. (2) Output image after opening where the 
smaller region was removed because it was smaller 
1 5 than the structuring element. 



FILTER DEEP REGIONS - 554 

This filtering step is used to remove regions from a segmented image that start at a 
20 location deeper than a user specified depth threshold. This filtering is illustrated in Figure Q, 
where the input image contains two regions one region starts at row 2 and the other region 
starts at row 14. For each region on the input image, the starting row is determined after 
doing a connected component labeling - then using a depth threshold of 12, the regions 
where the starting row depth is greater than the depth threshold are removed from the input 
25 image. 
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5 10 15 20 5 10 15 20 

Figure Q: (1) Input image containing two foreground 
regions. (2) Output image after removing the region 
whose starting row is greater than the depth threshold 
of 12. 



HEAD DIAMETER RANGE - 730 

For a gestational age of 20 weeks, the bi-parietal diameter (BPD) is typically 46 mm. 
The calculated Head Diameter search range, which ranges from -30% to +30% of the typical 
BPD value, is found to be: 

32.2 36.8 41.4 46.0 50.6 55.2 59.8 

For a gestational age of 36 weeks, the bi-parietal diameter (BPD) is typically 88 mm. 
The calculated Head Diameter search range, which ranges from -30% to +30% of the typical 
BPD value, is found to be: 

61.6 70.4 79.2 88.0 96.8 105.6 114.4 



POLAR HOUGH TRANSFORM - 738 

To illustrate the polar Hough transform, an exemplary image is generated of a circle 
in the Cartesian coordinates using the circle equations, E5. This circle image was then dilated 
to result in a disk like shape and then a few rows of data were set to zero. This input image is 
shown in Figure R(l). The goal of the circular Hough transform is to find the circle that best 
fits the white pixels in this input image. 

Next, this Cartesian input image was converted to polar domain by inverse scan 
conversion and the resulting image is shown in Figure R(2). To this polar coordinate image, 

_ 77 . Black Lowe & Graham— 

^ 

9 S ^ 1 5 DXUC-1-1020AP 816 Second Avenue 

A ^ Seattle, Washington 98104 

PATENT TRADEMARK OFFICE 206.381.3300 • F: 206.381.3301 



the polar Hough transform for a circle was applied, where for each white pixel on the image a 
polar circle of a known radius was drawn. The output of the Hough transform is the sum of 
all such circles and this is shown in Figure R(3). The center of the best-fit circle has the most 
points passing through it. So, to determine the best-fit circle, we simply found the pixel on 
5 the Hough output with the maximum value. The point with the maximum value represents 
the center of the best- fit circle. 
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(1) (2) 
Figure R: (1) The input Cartesian coordinate partial 
circle (2) The partial circle converted to polar 
coordinates, (3) The Hough transform output 
showing the sum of polar circles drawn around each 
white pixel of the input image. 



(3) 
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The best-fit circle determined by this Hough transform procedure is overlaid on the 
polar coordinate image and the Cartesian coordinate image and the two drawings are shown 
in Figure S. 





20 
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Figure S: The best fit circle found using a Polar 
coordinate Hough transform (1) overlaid on the polar 
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coordinate input data and (2) overlaid on the 
Cartesian coordinate input data. 



FILL CIRCLE REGION - 746 
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A circle region in Cartesian space has a somewhat different shape in the Polar 
coordinate space. To illustrate this polar coordinate circle, we use both Cartesian coordinate 
and polar coordinate circles. First, a Cartesian coordinate circle of radius 50 pixels and a 
center (120,250) is drawn using the circle equation, E5, (x-x 0 ) 2 + (y-y 0 ) 2 = R 2 ~ this 
circle is shown in Figure T( 1 ). 
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Now, the Cartesian center coordinates (x Q ,y 0 ) are converted from Cartesian to polar 
center (r 0 , y 0 ) using the scan conversion equations: 

r 0 2 =( x o 2 +yo 2 )> 
<f> 0 = tm l (y 0 /x 0 ) 

Next, the polar coordinate circle drawing and filling equation, E6, 
(rsin^-r o sin0 o ) 2 +(rcos0-r o cos0 o ) 2 = R 2 , 

is applied to draw and fill the same circle of radius R=50, in polar coordinates. This 
circle in the polar coordinates is shown in Figure T(2). 

Finally, to verify that this polar coordinate circle in indeed a representation of the 
Cartesian coordinate circle, the polar circle image is scan-converted to generate the 
representative Cartesian coordinate circle image. This scan-converted circle is shown in 
Figure T(3). Comparing Figure T(l) and Figure T(3), it can be seen that while the two 
circles are not identical, they are fairly similar. The differences are due to the fact that the 
polar coordinate data cannot represent Cartesian data completely. 
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Figure T Polar coordinate circle filling. (1) A filled 
circle in a Cartesian coordinate image (2) The 
equivalent filled circle in polar coordinate image, (3) 
The scan converted polar coordinate filled circle 
image. 
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